Integrated electronic and photonic backplane architecture for display panels

ABSTRACT

In various embodiments, an apparatus comprises a composite backplane that modulates light from a light source, where the composite backplane comprises an electronics layer disposed on a substrate, a photonics integrated circuit (IC) layer disposed on the electronics layer that causes light from the light source to propagate in a first direction, and an active light modulation (ALM) interface layer disposed on the photonics IC layer controls an ALM interface layer in order to control the light propagating in the first direction.

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relates generally to videodisplays, head-mounted displays and, more specifically, to an integratedelectronics and photonic backplane architecture for display panels.

Description of the Related Art

A display assembly, included in optical assemblies such as head-mounteddisplays, mobile displays, and so forth, operates by using pixels tocontrol light wavelengths that propagate to a lens. The lensconcentrates light wavelengths provided by the display assembly to reachthe eye of the user as an image. For example, a virtual reality displayassembly includes a thin-film transistor (TFT) liquid crystal display(LCD) panel disposed on a backlight unit (BLU) that acts as a lightsource for the TFT LCD panel.

In some examples, each of the LCD panel and the backlight unit ispackaged with a cover glass. When the LCD panel is disposed on the coverglass of the backlight unit, the cover glass over the backlight unitcreates a large distance between the backlight unit and the LCD panel.At least one drawback of conventional display assemblies is that, whenthe light source, such as a laser source or a light-emitting diode (LED)array, is randomly scattered by the backlight unit, the directionalityof the light scattered by the backlight unit becomes large anduncontrollable. This leads to a low photon efficiency and limited fillfactor for the amount of light that the display panel provides. Further,the large distances between layers result in bulky designs that limithow such display assemblies can be included in compact opticalassemblies.

Some display assemblies include a photonic integrated circuit(PIC)-based backlight unit in order to precisely control the emissioncone of a given light source on a pixel-by-pixel basis, improving thephoton efficiency of the backlight unit. However, such displayassemblies require post-fabrication alignment at a high precision, whichis costly and time-consuming. These PIC-based backlight units alsoinclude cover glass over the LCD panel and backlight unit, whichsimilarly cause the assembly to be bulky. Moreover, the distance betweenthe PIC-based backlight unit and the LCD panel causes a large amount ofcrosstalk between neighboring pixels.

SUMMARY

In various embodiments, an apparatus comprises a composite backplanethat modulates light from a light source, the composite backplanecomprising an electronics layer disposed on a substrate, a photonicsintegrated circuit (IC) layer disposed on the electronics layer thatcauses light from the light source to propagate in a first direction,and an active light modulation (ALM) interface layer disposed on thephotonics IC layer controls an active medium layer in order to controlthe light propagating in the first direction.

Other embodiments include a display system comprising a display panelcomprising a composite backplane, including an electronics layerdisposed on a substrate, a photonics integrated circuit (IC) layerdisposed on the electronics layer that directs light from a light sourceto propagate in a first direction, and an active light modulation (ALM)interface layer disposed the photonics IC layer, an active medium layerdisposed on the ALM interface layer comprising sets of pixels includingsets of an active media, and a top cover layer, and a controller causingthe display panel to modify the light controlled via the active mediumlayer or the photonic IC layer.

At least one technical advantage of the disclosed embodiments relativeto the prior art is that that composite backplane comprising a photonicintegrated circuit layer disposed between an electronic IC and a liquidcrystal interface layer enables the composite backplane to possess acompact composition that support various types of light sources (e.g.,lasers, light-emitting diodes, etc.) and provides high efficiency andhigh pixel density. Further, the composite backplane can be fabricatedusing various lithographic fabrication processes and can be included ina wide range of display assemblies.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the variousembodiments can be understood in detail, a more particular descriptionof the inventive concepts, briefly summarized above, may be had byreference to various embodiments, some of which are illustrated in theappended drawings. It is to be noted, however, that the appendeddrawings illustrate only typical embodiments of the inventive conceptsand are therefore not to be considered limiting of scope in any way, andthat there are other equally effective embodiments.

FIG. 1 illustrates a die fabrication system configured to implement oneor more aspects of the present disclosure.

FIG. 2 illustrates a view of a composite backplane processed by thedevice fabrication system of FIG. 1 , according to various embodimentsof the present disclosure.

FIG. 3 illustrates views of various configurations for a liquid crystalinterface layer included in a composite backplane of FIG. 2 , accordingto various embodiments of the present disclosure

FIGS. 4A-4B illustrates views of various configurations for a photonicintegrated circuit layer included in the composite backplane of FIG. 2 ,according to various embodiments of the present disclosure.

FIGS. 5A-5B illustrates views of various configurations for anelectronics integrated circuit layer included in the composite backplaneof FIG. 2 , according to various embodiments of the present disclosure.

FIGS. 6A-6C illustrates views of various configurations for a liquidcrystal cell 600 including the composite backplane of FIG. 2 , accordingto various embodiments of the present disclosure.

FIG. 7 sets forth a flow diagram of method steps for fabricating acomposite backplane for a display panel, according to the variousembodiments of the present disclosure.

FIG. 8 is a block diagram of an embodiment of a near-eye display (NED)system in which a console operates, according to various embodiments.

FIG. 9A is a diagram of an NED, according to various embodiments.

FIG. 9B is another diagram of an NED, according to various embodiments.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth toprovide a more thorough understanding of the various embodiments.However, it will be apparent to one skilled in the art that theinventive concepts may be practiced without one or more of thesespecific details.

Overview

As described above, one problem with conventional approaches for displaypanels is that the composition of layers needed to provide a qualityimage causes various wavelengths of light to be blocked or propagate indirections other than towards the lens of a display assembly, resultingin large losses in efficiency associated with providing light, such aslow pixel efficiency and a low fill factor. Further, various techniquesto raise the efficiency of the display panel cause the fabricationprocess to become more complex. For example, the process may requirespecialized fabrication and calibration equipment in order to properlyalign pixels in a given display panel, significantly increasing the costand complexity associated with including the display panel as acomponent in display assemblies.

Accordingly, in various embodiments disclosed herein, a die fabricationsystem produces a composite backplane that includes a specific set oflayers to control light provided from a light source. The diefabrication system disposes a set of layers in sequence to produce acomposite backplane for a liquid crystal unit in order to drive adisplay panel that is compact, highly-efficient, and possesses a highpixel density. The composite backplane includes an electronic IC layer,a photonic integrated circuit (PIC) layer, and an active lightmodulation (ALM) interface layer over a substrate that directs light toa set of pixels in a separate active medium layer.

The composite backplane can be included in both front-lit display panelsand back-lit display panels and can support various light sources (e.g.,lasers, light-emitting diodes, etc.) and provides a high color gamut.Further, the composite backplane can be fabricated using variouslithographic fabrication processes to produce a compact composition thatcan be included in a wide range of display assemblies. Such assembliescan be used in systems that control the light provided via amplitudemodulation (e.g., two-dimensional display panels) or via phasemodulation (e.g., coherent holographic display panels).

System Overview

FIG. 1 illustrates a die fabrication system 100 configured to implementone or more aspects of the present disclosure. As shown, die fabricationsystem 100 includes a wafer 102, a fabrication system 110, and etchedbackplanes 120. The fabrication system 110 includes a projection lens112, a photomask 114, and a wafer loader 116. The etched backplane 120includes composite backplane 121 that includes an active lightmodulation (ALM) interface layer 122, a photonic integrated circuit (IC)layer 124, an electronic IC layer 126, and a substrate 128.

In operation, the fabrication system 110 causes the wafer loader 116 tomanipulate the wafer 102 via moving, rotating, slicing, etc. Thefabrication system 110 also causes the projection lens 112 and photomask114 to pattern portions of the wafer 102 to generate the etchedbackplane 120. In various embodiments, the fabrication system 110 usesvarious lithography-based nano-manufacturing processes associated withfabricating electronic components and/or photonic components in order topattern the etched backplanes 120 on the wafer 102. In some embodiments,the fabrication system 110 separates the etched backplanes 120 intoseparate die packages, where each die package includes a compositebackplane 121. In such instances, the respective composite backplanesmay be combined with other layers and/or components to fabricate displayassemblies, such as a display panel, a device containing a display panel(e.g., a mobile phone, tablet, wearable near-eye display, etc.), and soforth.

Upon fabrication, a display panel including the composite backplane 121may be thinner than other display panels due to reduced gaps betweenrespective layers of the composite backplane 121. For example, a gapbetween the ALM interface layer 122 and the photonic IC layer 124 may bereduced compared to other techniques, as the composite backplane 121does not include a cover glass over the photonics IC layer 124. Further,the thickness of the composite backplane 121 could be reduced byincluding electronic modules for controlling the respective ALMinterface layer 122 and the photonics IC layer 124 in the electronic IC126. In such instances, a composite backplane 121 having a compositionthat includes the ALM interface layer 122, the photonics IC layer 124and the electronic IC 126 in this manner enables the composite backplane121 to provide a better fill factor, defined as a ratio of the lightemitting area of each pixel to the surface area occupied by the pixel,for the display panel.

The wafer 102 may be a semiconducting material, such as silicon, that isused to fabricate an IC die package. In various embodiments, thefabrication system 110 may perform various techniques on the wafer 102in order to generate the etched backplanes 120. For example, thefabrication system 110 could perform wafer-level packaging (WLP)techniques to pattern and dice the wafer 102 in order to produce the ICdie packages. In some embodiments, the fabrication system 110 couldperform panel-level packaging techniques to generate a panel-sizedcomposite backplane 121 to drive a display panel of a specified size.Additionally or alternatively, in some embodiments, the fabricationsystem 110 may perform other techniques (e.g., flip chip packaging,quilt packaging, etc.) to prepare the etched backplanes 120.

The fabrication system 110 includes one or more devices that patternand/or slice portions of the wafer 102 in order to generate the etchedbackplanes 120. For example, the fabrication system 110 could includeone or more devices, such as one or more wafer loaders 116 (e.g., 116 a,116 b, etc.), photomasks 114 (e.g., 114 a, 114 b, etc.), projectionlenses 112 (e.g., 112 a, 112 b, etc.) and/or other devices (e.g.,grinders, coaters, developers, etchers, strippers, etc.) that performvarious processes to pattern the surface of the wafer 102. For example,the fabrication system 110 could use the projection lens 112 and thephotomask 114 in conjunction to form a given layer of the etchedbackplanes 120. In such instances, the fabrication system 110 could usedifferent projection lenses 112 and/or photomasks 114 to form thedifferent layers included in the composite backplane 121. In someembodiments, the fabrication system 110 could also include devices thatperform various functions to form layers on the wafer 102. In suchinstances, the fabrication system 110 could perform a technique, such aschemical mechanical planarization (CMP), that adds layers that form agiven substrate and/or layer on the surface of the wafer 102. In someembodiments, one or more devices included in the fabrication system 110may perform other processes in the fabrication process. For example, thewafer loader 116 could grind and polish a surface of the wafer 102before proceeding to add a given layer onto the wafer 102.

The processing unit 108 includes one or more processors that control theoperation of the fabrication system 110. In various embodiments, theprocessing unit 108 may be one or more central processing units (CPUs),multi-core processors, microprocessors, microcontrollers, digital signalprocessors, field-programmable gate arrays (FPGAs), application-specificintegrated circuits (ASICs), and/or the like. In some embodiments, theprocessing unit 108 may be included as part of an operator workstationand/or operated separately from, but in coordination with, the operatorworkstation. The memory 104 may be used to store software executed bythe processing unit 108. The memory 104 may also store one or more datastructures used during the operation of the fabrication system 110. Thememory 104 may include one or more types of machine-readable media. Somecommon forms of machine-readable media may include floppy disk, flexibledisk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, anyother optical medium, punch cards, paper tape, any other physical mediumwith patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memorychip or cartridge, and/or any other medium from which a processor orcomputer is adapted to read.

The processing unit 108 executes the fabrication application 106 toproduce etched backplanes 120 via control of one or more devices in thefabrication system 110 (e.g., the projection lens 112, the photomask114, the wafer loader 116, etc.). In various embodiments, thefabrication application 106 may receive inputs specifying configurationsfor each of the layers 122-126 included in the composite backplane 121.The fabrication application 106 may cause the one or more devices in thefabrication system 110 to pattern layers onto the wafer 102 in order toproduce the etched backplanes 120. For example, the fabricationapplication 106 could receive a device design as an input that specifiesthe configuration of components in each of the layers 122-126. In suchinstances, the processing unit 108 could execute the fabricationapplication 106 to control the fabrication system 110 when patterningsuccessive layers of the etched backplanes 120.

In some embodiments, the fabrication application 106 may optimize theconfiguration of one or more layers 122-126 based on a targetconfiguration. For example, an operator could identify one or moreseparate characteristics (e.g., light source type, light sourceposition, modulation type, etc.) of a device that will include thecomposite backplane 121. In such instances, the fabrication application106 could determine the components to include in each respective layer122-126 and position components within each layer 122-126 in anarrangement that enables the composite backplane 121 to possess thecharacteristics specified in the target configuration.

The etched backplanes 120 include one or more portions of a compositebackplane 121 that are used to drive the operation of a display panel.In some embodiments, the etched backplanes 121 are a patterned andetched version of the wafer 102 and include backplanes for multiplepackages (e.g., multiple display panels). In some embodiments, thefabrication system 110 may slice the wafer into separate panels, wherethe etched backplanes 120 include a set of separate composite backplanes121.

A composite backplane 121 is a version of a display device backplanethat provides structural support for a display panel. As shown, thecomposite backplane 121 includes a composition of layers that combine todrive and control light provided by a light source and modulated by aset of liquid crystals in a separate layer. For example, a display panelincluding a layer of liquid crystals (not shown) and the compositebackplane 121 may use one or more electrodes included in the ALMinterface layer 122 to control the orientation of subsets of liquidcrystals in order to control the polarity of light rays passing throughthe liquid crystal layer.

For example, the composite backplane 121 could compose three layers thatare fabricated in sequence using standard lithographic manufacturingprocesses. In this example, the layers include the ALM interface layer122 that includes a set of pixelated conducting pads (e.g., electrodes)for liquid crystal cells; the ALM interface layer 122 could also includea black matrix layer, a reflection control coating (e.g.,anti-reflection coating, partial-reflection coating, high-reflectioncoating, etc.). The layers also include the photonic IC layer 124 thatincludes light-guiding waveguides and out-coupling components insingle-layer or multi-layer configurations. The layers can also includethe electronic IC layer 126 that includes electronic bus lines forpower, control & data, as well as integrated electronic circuitry forcomponents in the ALM interface layer 122 and the photonic IC layer 124.In some embodiments, one or more vertical metallic via paths (“vias”)are used to connect the electronic IC layer 126 with the ALM interfacelayer 122 and the photonic IC layer 124, respectively.

FIG. 2 illustrates a view of a composite backplane 200 processed by thedevice fabrication system 100 of FIG. 1 , according to variousembodiments of the present disclosure. As shown, the composite backplane200 includes the ALM interface layer 122, the photonic IC layer 124, theelectronic IC layer 126, and the substrate 128. The ALM interface layer122 includes pixelated electrodes 220, a black matrix layer 217, and analignment layer 218. The photonic IC layer includes light-guidingwaveguides 214 and output couplers 215. The electronic IC layer 126includes electronic ICs 212, metallic vias 210, and electronic bus lines208.

As shown, the composite backplane 200 controls the light generated by alight source. In particular, the composite backplane 200 controls bothan active media, such as liquid crystals (not shown), via the pixelatedelectrodes 220 included in the ALM interface layer 122 and thelight-guiding waveguides (“photonic circuits”) 214, output couplers 215,and other light modulating and out-coupling components. In particular,the light-guiding waveguides 214 control the light 206 provided by alight source, such as a side light source or other light source (e.g.,front-lit light source, back-light lit source) that provides the lightto the photonic IC layer. In addition, the pixelated electrodes in theALM interface layer 122 control sets of active media in a separateactive medium layer that modify the polarization of the light providedby the light source.

Controlling the light propagating from the light source through multiplecomponents increases the light efficiency (e.g., photon efficiency,pixel density, etc.) due to reductions in optical crosstalk betweenneighboring pixels and greater control of emission cones for the light.The composite backplane 200 provides benefits associated with bothreflective designs that modulate front-lit light sources andtransmissive designs that modulate back-lit light sources. Further, theconfiguration of the composite backplane 121 increases the fill factordue to reductions in components between layers, such as moving mostelectronic components to a separate electronic IC 126 and connecting theelectronic components with other components using metallic via paths210.

In some embodiments, the composite backplane 200 may operate inconjunction with a front-lit light source, such as a liquid crystal onsilicon (LCOS) layer disposed on the composite backplane 120. In suchinstances, a display panel including the LCOS layer and the compositebackplane may act as a spatial light modulator and may includeadditional components in the layers 122-126 in order to separate theincident and reflected light paths of light generated by the lightsource in the LCOS layer. In other embodiments, the composite backplane200 may operate in conjunction with a back-lit light source and act as aspatial light modulator for a transmissive liquid crystal layer.

In various embodiments, the composite backplane 200 may be integratedwith other devices or components. For example, the composite backplane200 could be integrated with one or more on-chip light sources, such aslaser sources, superluminescent light-emitting diode (SLED) sourcesand/or light-emitting diode (LED) arrays. In such cases, the device mayact as a stand-alone display module that modulates light produced by theon-chip light sources. In various embodiments, the light source can beany type of LED (e.g., LED, pLED, organic light-emitting diode (OLED),quantum-dot light-emitting diode (QDLED), perovskite light-emittingdiode (PeLED), etc.). In some embodiments, the light source can be oneor more lasers, such as diode lasers, vertical cavity surface emissionlasers, heterogeneously-integrated lasers, hybrid lasers, fiber lasers,and so forth. In some embodiments, the light source may be a nonlinearlight source from one or more other light sources, such as a pump laserfield, a sum-frequency generation source, second-harmonic generationsource, a four-wave-mixing source, a difference-frequency generationsource, a parametric down-conversion source, and so forth.

Additionally or alternatively, the composite backplane 200 may beintegrated with additional integrated circuit modules in order to enablemore power-efficient data processing and transferring of data betweenthe processor and the light modulating components in the layers 122-124.

Variations for the Composite Backplane Architecture

FIG. 3 illustrates views of various configurations for a liquid crystalinterface layer included in a composite backplane 200 of FIG. 2 ,according to various embodiments of the present disclosure. As shown, aset of candidate ALM interface layers 300 include candidate ALMinterface layer configurations 310-330.

As shown, each configuration 310-330 of the ALM interface layer 122 canbe combined independently with other configurations for the respectivephotonics IC layer 124 and/or the electronic IC 126. In variousembodiments, each of the candidate ALM interface layers 300 may act asan active pixel interface layer that includes components to control oneor more ALM pixel cells for a given active medium (e.g., a set of liquidcrystals in an active media layer that control light for one pixel). TheALM interface layer 300 may also include different types of opticalcoating (e.g., AR or PR coating 216), alignment layers to position theelectrodes, and/or black matrix layers that block extraneous lightbetween pixels.

For example, configuration 310 could include a set of pixelatedelectrodes 220 to control separate active media pixel cells (not shown).In such instances, the pixelated electrodes can provide individualelectronic control signals to separate pixel cells. The configuration310 may include an anti-reflective (AR) coating 216 for single-passoperation of the pixel cells, or partial-reflective (PR) coating 216 forresonance-mode operation of the pixel cells.

In various embodiments, the candidate ALM interface layers 300 mayinclude an alignment layer 218. In some embodiments, the structure ofthe alignment layer 218 may be based on a micro-structured surface thatcan be lithographically fabricated, or the structure may be based onmaterials that can be spun on top of the device. In some embodiments,the alignment layer 218 fills gaps between components in the layer. Forexample, configuration 310 illustrates the alignment layer occupyingspaces between the pixelated electrodes 220.

In some embodiments, the candidate ALM interface layers 300 may includeblack matrix layers 217. The black matrix layer 217 may be located atvarious vertical positions within the ALM interface layer 122 and/or thephotonics IC layer 124. For example, the black matrix layer 217 could bepositioned in spaces between the pixelated electrodes 220 and may bemade of a reflective metallic material. When the black matrix layer 217is fabricated with the other components in the ALM interface layer 300,the black matrix layer 217 is positioned correctly and nopost-fabrication alignment is necessary. In some embodiments, the blackmatrix layer may reduce crosstalk between pixel cells by blocking orreflecting light that would otherwise propagate through the ALMinterface layer 300. In such instances, the black matrix layer 217reduces the amount of light seen in order to restrict furtherrefraction. In some embodiments, the configuration of the black matrixlayer 217 may differ. For example, configuration 320 includes a blackmatrix layer 217 that includes a thin black matrix film below eachelectrode 220 and thicker sections between each electrode 220. Inconfiguration 330, the black matrix layer 217 is also included in theoptical coating layer 216.

In various embodiments, a particular configuration 310-330 may beselected based on a set of design, fabrication, and/or operatingcharacteristics. For example, a designer could select specific materialsand fabrication processes for the ALM interface layer 300 in order to becompatible with the fabrication of the photonic IC layer 124 and/or theelectronic IC layer 126 (e.g., maximum processing temperature, materialcompatibility, etc.). In another example, a designer may specify theconfiguration of the electrodes and/or the black matrix layer in orderto control a greater number of pixels or to suppress opticalinefficiencies like scattering and crosstalk.

FIGS. 4A-4B illustrate views of various configurations for a photonicintegrated circuit layer included in the composite backplane 200 of FIG.2 , according to various embodiments of the present disclosure. As shownin FIG. 4A, a set of candidate photonic IC layers 400 include candidatephotonic IC layer configurations 410-430; as shown in FIG. 4B, a secondset of candidate photonic IC layers 440 include multi-layer photonic IClayer configurations 450-470.

In various embodiments, each of the candidate photonic IC layers 400,440 may include a single layer or multiple layers of photonic integratedcircuits (PICs) 214 embedded in a substrate. In operation, the PICs 214may include color multiplexers (MUX), color demultiplexers (DEMUX),waveguides, couplers, splitters, active light modulating components,and/or out-coupling components. The active PIC components can includeamplitude modulators, phase modulators, polarization modulators, etc.that provide active control in the property of the emitted light. Eachof the active PIC components can be driven by electronic circuitry builtin the electronic IC layer and connected with vertical metallic vias.Each active PIC component 214 can perform various operations on incominglight, such as focusing, splitting, isolation, polarization modulation,coupling, amplitude modulation and/or phase modulation.

In various embodiments, some of the PIC components may be light-couplingcomponents that connect the PICs 214 to the light source. For example,the light-coupling components may be optical fibers, light guides,waveguides, nanowires, microwires, lenses, waveguide-grating couplers,waveguide mode converter, lensed fibers, metalenses, plasmonics-basedcouplers, and so forth. In various embodiments, some of the PICcomponents may be out-coupling components, such as output couplers 215that couple light from the PICs 214 and turn the wavelengths of light206 into free space in a specific direction (e.g., direct the lightvertically towards the active medium layer). Additionally oralternatively, the out-coupling components may include waveguide gratingcouplers, ring resonators, side-coupled scatterers, top-coupledscatterers, etc. The gratings can contain multi-material-layers,multi-etch-depth, straight or slanted, or any combination.

In some embodiments, a given configuration 420, 430 may include anadditional layer, such as a dielectric high-reflective (HR) coating 422or a metallic film 432 beneath the PICs 214 in order to improve theout-coupling efficiency of the output couplers 215 by reflecting some ofthe light towards the pixel cells. Additionally or alternatively, aparticular configuration 410-430, 450-470 may be selected based on a setof design, fabrication, and/or operating characteristics. For example, adesigner could select specific materials and fabrication processes forthe photonics IC layer 400, 440 in order to be compatible with thefabrication of the ALM interface layer 122 and/or the electronic IClayer 126 (e.g., maximum processing temperature, material compatibility,etc.).

In various embodiments, as shown in FIG. 4B, some configurations 450-470may include multiple layers of PICs 214 (e.g., 214 a, 214 b, 214 c,etc.). In such instances, the light 206 a from the light source may becoupled (as illustrated by 206 b) between a first layer of waveguides214 a and a second layer of waveguides 214 b. In some embodiments, theconfiguration 460 may include an active intensity modulator 452 thatdynamically controls properties of the light 206. In some embodiments,the photonic IC layer 440 may include various PIC layers 214 (e.g., 1-6layers) of varying lengths. For example, configurations 450, 460 includetwo PIC layers 214 and configuration 470 includes three PIC layers 214and multiple active intensity modulators 452.

FIGS. 5A-5B illustrate views of various configurations for anelectronics integrated circuit (IC) layer included in the compositebackplane of FIG. 2 , according to various embodiments of the presentdisclosure. As shown, a set of candidate electronics IC layers 500include candidate electronic IC layer configurations 510-540.

In various embodiments, each of the configurations 510-540 mayincorporate complementary metal-oxide-semiconductor (CMOS) technology orother semiconductor IC technology (e.g., PMOS and NMOS). The electronicmodules included in the electronics IC layer 500 may be based on Siliconwafers (similar to LCOS technology), or using TFT technology (α-Si,low-temperature polycrystalline silicon (LTPS), Organic-TFT, oxide-TFT,LTPS+oxide-TFT, etc.) on transparent substrates.

In various embodiments, the electronic IC layer 500 may contain power,data, and other electronic bus lines, as well as circuitry for eachactive pixel cell to drive a set of active media included in the pixelcell. For example, configuration 530 includes a set of electronic ICmodules 212 a, 212 b, 212 c that drive separate pixelated electrodes220. Each electronic module 212 is connected to the component in usingmetallic via paths 210 that may be positioned between components, suchas metallic vias being positioned between the PIC components 214 in thephotonic IC layer 124 in order to connect to the pixelated electrodes220 in the ALM interface layer 122. In another example, theconfiguration 540 includes one or more electronic modules 212 d for thePIC components (e.g., active PIC components) in the photonic IC layer124. The electronic module 212 d is connected to the active PICcomponent through metallic vias that connect to the active PIC componentin the photonic IC layer 124.

In some embodiments, the electronics IC layer 500 may contain additionalcircuitry (e.g., metallic vias 210, electronic bus lines 208) and/oradditional electronic modules 212 e. For example, the configuration 540includes additional electronic modules (e.g., modules 212 e for the ALMinterface layer 122, modules 212 f for the photonics IC layer 124) inorder to provide on-chip processing of data for a display panel. Theadditional electronic modules 212 e, 212 f could improve the powerconsumption, data processing, transfer speed by transmitting commands toother electronic ICs 212 that control the components in the other layers122-124.

FIGS. 6A-6C illustrate views of various configurations for a cell 600including the composite backplane 200 of FIG. 2 , according to variousembodiments of the present disclosure. As shown in FIG. 6A, the cell 600includes the substrate 128, the electronics IC layer 126, the photonicsIC layer 124, the ALM interface layer 122, an active medium layer 620,and top layer 610. In some embodiments, the top layer 610 may include anoptical coating 602 (e.g., an AR coating), a polarizer layer 604, anelectrostatic shielding layer 606, a top substrate 608, an electrodelayer 614 and/or an alignment layer 614. Although cell 600 is shown withliquid crystals as the active media, the cell 600 can include otheractive media in lieu of liquid crystals in the active media layer 620.

In operation, the pixelated electrodes 220 control separate sets ofactive media, such as liquid crystals 622 included in the active mediumlayer 620, in order to control the light output from the cell 600. Forexample, the pixelated electrodes 220 could separately modulate theamplitude of separate subsets of liquid crystals 622 in order to modifythe amplitude of light output from the cell 600. In other embodiments,the pixelated electrodes 220 may control other types of active media.

As shown in FIG. 6B, the pixelated electrodes 220 could separatelymodulate the phase of separate subsets of liquid crystals 622 in orderto modify the light output from the cell 640. In such instances, the toplayer 610 may not include the polarizer layer 604. The respectiveportions 644, 646 of the light 206 output by the cell 622 may havedifferent polarizations based on the polarization of the differentsubsets of the liquid crystals 622.

As shown in FIG. 6C, the portion of a pixel cell illustrates a top view660 and an isometric view 680 of portions of the composite backplane 200included in the cells 600, 640. As shown in the respective top andisometric views 660, 680, the metallic vias 210 and electronic buses 208are positioned in locations to provide connections to other componentsin other layers (e.g., the output couplers 215, the pixelated electrodes220) and minimize obstruction of wavelengths of light 206 as the lightpropagates through the composite backplane 200.

FIG. 7 sets forth a flow diagram of method steps 700 for fabricating acomposite backplane for a display panel, according to the variousembodiments of the present disclosure. Although the method steps aredescribed in conjunction with FIGS. 1-6C, persons of ordinary skill inthe art will understand that any system configured to perform thismethod and/or methods described herein, in any order, and in anycombination not logically contradicted, is within the scope of thepresent disclosure.

As shown, the method 700 begins at step 702, where the fabricationsystem 110 determines a configuration for the electronics IC layer 126.In various embodiments, the fabrication application 106 included in thefabrication system 110 may receive indications of a specificconfiguration of components (e.g., electrodes 220) used to control setsof active media in a separate active media layer 620. In someembodiments, the configuration may include additional characteristicsand/or components, such as the location of the black matrix layer 217,optical coating layer 216, and/or the alignment layer 218.

In some embodiments, the fabrication application 106 may optimize theconfiguration of one or more layers 122-126 based on a targetconfiguration. For example, an operator could identify one or moreseparate characteristics (e.g., light source type, light sourceposition, modulation type, etc.) of a device that will include thecomposite backplane 121. In such instances, the fabrication application106 could determine the components to include in each respective layer122-126 and position components within each layer 122-126 in anarrangement that enables the composite backplane 121 to possess thecharacteristics specified in the target configuration.

At step 704, the fabrication system 110 determines a configuration forthe photonics IC layer 124. In various embodiments, the fabricationapplication 106 may receive indications of a specific configuration ofcomponents, such as specific photonic circuits 214, output couplers 215,and/or additional layers 422, 432 to include in the photonic IC layer124. The photonic layer 124 uses the specified components to control thewavelengths of light 206 from the light source. In some embodiments, thephotonic IC layer 124 includes active PIC components (e.g., amplitude,phase, and/or polarization modulators) that actively control the light206.

At step 706, the fabrication system 110 determines a configuration forthe electronics IC layer 126. In various embodiments, the fabricationapplication 106 may receive indications of a specific configuration ofcomponents to include in the electronics IC layer 126. In someembodiments, the components include specific integrated circuit modulesused to control the operation of components in other layers, such as theelectrodes 220 in the ALM interface layer 122 and/or the active PICcomponents in the photonic IC layer 124. In some embodiments, theelectronics IC layer 126 may include additional components, such asadditional bus lines 208 for power, metallic via paths 210 forcommunication, and/or additional electronic IC modules 212 for dataprocessing.

At step 708, the fabrication system 110 forms a composite backplane 200based on the determined layers. In various embodiments, the fabricationapplication 106 may produce a composite backplane 121 that includes eachof the determined layers 122-126. Upon determining the design for thecomposite backplane 121, the fabrication system 110 causes the waferloader 116 to manipulate the wafer 102 and uses the projection lens 112and photomask 114 to pattern portions of the wafer 102 to generateetched backplane 120 that includes multiple panels of the compositebackplane 121. In various embodiments, the fabrication system may usevarious lithography-based nano-manufacturing processes associated withfabricating electronic components and/or photonic components in order topattern various the etched backplanes 120 on the wafer 102.

At step 710, the fabrication system 110 fabricates a display panel thatincludes the composite backplane 200. In various embodiments, thefabrication system 100 combines the composite backplane 121 with otherlayers, such as the active medium layer 620, top layer 610, and/orcomponents to fabricate display assemblies, such as a display panel, adevice containing a display panel (e.g., a mobile phone, tablet,wearable near-eye display, etc.), and so forth. In various embodiments,the fabricated display panel does not require post-fabrication alignmentof the components included in the composite backplane 121.

The Artificial Reality System

Embodiments of the disclosure may include or be implemented inconjunction with an artificial reality system. Artificial reality is aform of reality that has been adjusted in some manner beforepresentation to a user, which may include, e.g., a virtual reality (VR),an augmented reality (AR), a mixed reality (MR), a hybrid reality, orsome combination and/or derivatives thereof. Artificial reality contentmay include completely generated content or generated content combinedwith captured (e.g., real-world) content. The artificial reality contentmay include video, audio, haptic feedback, or some combination thereof,and any of which may be presented in a single channel or in multiplechannels (such as stereo video that produces a three-dimensional effectto the viewer). Additionally, in some embodiments, artificial realitymay also be associated with applications, products, accessories,services, or some combination thereof, that are used to, e.g., createcontent in an artificial reality and/or are otherwise used in (e.g.,perform activities in) an artificial reality. The artificial realitysystem that provides the artificial reality content may be implementedon various platforms, including a head-mounted display (HMD) or near-eyedisplay (NED) connected to a host computer system, a standalone HMD orNED, a mobile device or computing system, or any other hardware platformcapable of providing artificial reality content to one or more viewers.

FIG. 8 is a block diagram of an embodiment of a near-eye display (NED)system 800 in which a console operates, according to variousembodiments. The NED system 800 may operate in a virtual reality (VR)system environment, an augmented reality (AR) system environment, amixed reality (MR) system environment, or some combination thereof. TheNED system 800 shown in FIG. 8 comprises a NED 805 and an input/output(I/O) interface 875 that is coupled to the console 870. In variousembodiments, the composite display system 800 is included in or operatesin conjunction with the NED system 800. For example, the compositedisplay system 800 may be included within NED 805 or may be coupled tothe console 870 and/or the NED

While FIG. 8 shows an example NED system 800 including one NED 805 andone I/O interface 875, in other embodiments any number of thesecomponents may be included in the NED system 800. For example, there maybe multiple NEDs 805, and each NED 805 has an associated I/O interface875. Each NED 805 and I/O interface 875 communicates with the console870. In alternative configurations, different and/or additionalcomponents may be included in the NED system 800. Additionally, variouscomponents included within the NED 805, the console 870, and the I/Ointerface 875 may be distributed in a different manner than is describedin conjunction with FIGS. 1-3B in some embodiments. For example, some orall of the functionality of the console 870 may be provided by the NED805 and vice versa.

The NED 805 may be a head-mounted display that presents content to auser. The content may include virtual and/or augmented views of aphysical, real-world environment including computer-generated elements(e.g., two-dimensional or three-dimensional images, two-dimensional orthree-dimensional video, sound, etc.). In some embodiments, the NED 805may also present audio content to a user. The NED 805 and/or the console870 may transmit the audio content to an external device via the I/Ointerface 875. The external device may include various forms of speakersystems and/or headphones. In various embodiments, the audio content issynchronized with visual content being displayed by the NED 805.

The NED 805 may comprise one or more rigid bodies, which may be rigidlyor non-rigidly coupled together. A rigid coupling between rigid bodiescauses the coupled rigid bodies to act as a single rigid entity. Incontrast, a non-rigid coupling between rigid bodies allows the rigidbodies to move relative to each other.

As shown in FIG. 8 , the NED 805 may include a depth camera assembly(DCA) 855, one or more locators 820, a display 825, an optical assembly830, one or more position sensors 835, an inertial measurement unit(IMU) 840, an eye tracking system 845, and a varifocal module 850. Insome embodiments, the display 825 and the optical assembly 830 can beintegrated together into a projection assembly. Various embodiments ofthe NED 805 may have additional, fewer, or different components thanthose listed above. Additionally, the functionality of each componentmay be partially or completely encompassed by the functionality of oneor more other components in various embodiments.

The DCA 855 captures sensor data describing depth information of an areasurrounding the NED 805. The sensor data may be generated by one or acombination of depth imaging techniques, such as triangulation,structured light imaging, time-of-flight imaging, stereo imaging, laserscan, and so forth. The DCA 855 can compute various depth properties ofthe area surrounding the NED 805 using the sensor data. Additionally oralternatively, the DCA 855 may transmit the sensor data to the console870 for processing. Further, in various embodiments, the DCA 855captures or samples sensor data at different times. For example, the DCA855 could sample sensor data at different times within a time window toobtain sensor data along a time dimension.

The DCA 855 includes an illumination source, an imaging device, and acontroller. The illumination source emits light onto an area surroundingthe NED 805. In an embodiment, the emitted light is structured light.The illumination source includes a plurality of emitters that each emitslight having certain characteristics (e.g., wavelength, polarization,coherence, temporal behavior, etc.). The characteristics may be the sameor different between emitters, and the emitters can be operatedsimultaneously or individually. In one embodiment, the plurality ofemitters could be, e.g., laser diodes (such as edge emitters), inorganicor organic light-emitting diodes (LEDs), a vertical-cavitysurface-emitting laser (VCSEL), or some other source. In someembodiments, a single emitter or a plurality of emitters in theillumination source can emit light having a structured light pattern.The imaging device captures ambient light in the environment surroundingNED 805, in addition to light reflected off of objects in theenvironment that is generated by the plurality of emitters. In variousembodiments, the imaging device may be an infrared camera or a cameraconfigured to operate in a visible spectrum. The controller coordinateshow the illumination source emits light and how the imaging devicecaptures light. For example, the controller may determine a brightnessof the emitted light. In some embodiments, the controller also analyzesdetected light to detect objects in the environment and positioninformation related to those objects.

The locators 820 are objects located in specific positions on the NED805 relative to one another and relative to a specific reference pointon the NED 805. A locator 820 may be a light emitting diode (LED), acorner cube reflector, a reflective marker, a type of light source thatcontrasts with an environment in which the NED 805 operates, or somecombination thereof. In embodiments where the locators 820 are active(i.e., an LED or other type of light emitting device), the locators 820may emit light in the visible band (˜380 nm to 950 nm), in the infrared(IR) band (˜950 nm to 9700 nm), in the ultraviolet band (70 nm to 380nm), some other portion of the electromagnetic spectrum, or somecombination thereof.

In some embodiments, the locators 820 are located beneath an outersurface of the NED 805, which is transparent to the wavelengths of lightemitted or reflected by the locators 820 or is thin enough not tosubstantially attenuate the wavelengths of light emitted or reflected bythe locators 820. Additionally, in some embodiments, the outer surfaceor other portions of the NED 805 are opaque in the visible band ofwavelengths of light. Thus, the locators 820 may emit light in the IRband under an outer surface that is transparent in the IR band butopaque in the visible band.

The display 825 displays two-dimensional or three-dimensional images tothe user in accordance with pixel data received from the console 870and/or one or more other sources. In various embodiments, the display825 comprises a single display or multiple displays (e.g., separatedisplays for each eye of a user). In some embodiments, the display 825comprises a single or multiple waveguide displays. Light can be coupledinto the single or multiple waveguide displays via, e.g., a liquidcrystal display (LCD), an organic light emitting diode (OLED) display,an inorganic light emitting diode (ILED) display, an active-matrixorganic light-emitting diode (AMOLED) display, a transparent organiclight emitting diode (TOLED) display, a laser-based display, one or morewaveguides, other types of displays, a scanner, a one-dimensional array,and so forth. In addition, combinations of the display types may beincorporated in display 825 and used separately, in parallel, and/or incombination.

The optical assembly 830 magnifies image light received from the display825, corrects optical errors associated with the image light, andpresents the corrected image light to a user of the NED 805. The opticalassembly 830 includes a plurality of optical elements. For example, oneor more of the following optical elements may be included in the opticalassembly 830: an aperture, a Fresnel lens, a convex lens, a concavelens, a filter, a reflecting surface, or any other suitable opticalelement that deflects, reflects, refracts, and/or in some way altersimage light. Moreover, the optical assembly 830 may include combinationsof different optical elements. In some embodiments, one or more of theoptical elements in the optical assembly 830 may have one or morecoatings, such as partially reflective or antireflective coatings.

In some embodiments, the optical assembly 830 may be designed to correctone or more types of optical errors. Examples of optical errors includebarrel or pincushion distortions, longitudinal chromatic aberrations, ortransverse chromatic aberrations. Other types of optical errors mayfurther include spherical aberrations, chromatic aberrations or errorsdue to the lens field curvature, astigmatisms, in addition to othertypes of optical errors. In some embodiments, visual content transmittedto the display 825 is pre-distorted, and the optical assembly 830corrects the distortion as image light from the display 825 passesthrough various optical elements of the optical assembly 830. In someembodiments, optical elements of the optical assembly 830 are integratedinto the display 825 as a projection assembly that includes at least onewaveguide coupled with one or more optical elements.

The IMU 840 is an electronic device that generates data indicating aposition of the NED 805 based on measurement signals received from oneor more of the position sensors 835 and from depth information receivedfrom the DCA 855. In some embodiments of the NED 805, the IMU 840 may bea dedicated hardware component. In other embodiments, the IMU 840 may bea software component implemented in one or more processors.

In operation, a position sensor 835 generates one or more measurementsignals in response to a motion of the NED 805. Examples of positionsensors 835 include: one or more accelerometers, one or more gyroscopes,one or more magnetometers, one or more altimeters, one or moreinclinometers, and/or various types of sensors for motion detection,drift detection, and/or error detection. The position sensors 835 may belocated external to the IMU 840, internal to the IMU 840, or somecombination thereof.

Based on the one or more measurement signals from one or more positionsensors 835, the IMU 840 generates data indicating an estimated currentposition of the NED 805 relative to an initial position of the NED 805.For example, the position sensors 835 include multiple accelerometers tomeasure translational motion (forward/back, up/down, left/right) andmultiple gyroscopes to measure rotational motion (e.g., pitch, yaw, androll). In some embodiments, the IMU 840 rapidly samples the measurementsignals and calculates the estimated current position of the NED 805from the sampled data. For example, the IMU 840 integrates themeasurement signals received from the accelerometers over time toestimate a velocity vector and integrates the velocity vector over timeto determine an estimated current position of a reference point on theNED 805. Alternatively, the IMU 840 provides the sampled measurementsignals to the console 870, which analyzes the sample data to determineone or more measurement errors. The console 870 may further transmit oneor more of control signals and/or measurement errors to the IMU 840 toconfigure the IMU 840 to correct and/or reduce one or more measurementerrors (e.g., drift errors). The reference point is a point that may beused to describe the position of the NED 805. The reference point maygenerally be defined as a point in space or a position related to aposition and/or orientation of the NED 805.

In various embodiments, the IMU 840 receives one or more parameters fromthe console 870. The one or more parameters are used to maintaintracking of the NED 805. Based on a received parameter, the IMU 840 mayadjust one or more IMU parameters (e.g., a sample rate). In someembodiments, certain parameters cause the IMU 840 to update an initialposition of the reference point so that it corresponds to a nextposition of the reference point. Updating the initial position of thereference point as the next calibrated position of the reference pointhelps reduce drift errors in detecting a current position estimate ofthe IMU 840.

In various embodiments, the eye tracking system 845 is integrated intothe NED 805. The eye-tracking system 845 may comprise one or moreillumination sources (e.g., infrared illumination source, visible lightillumination source) and one or more imaging devices (e.g., one or morecameras). In operation, the eye tracking system 845 generates andanalyzes tracking data related to a user's eyes as the user wears theNED 805. In various embodiments, the eye tracking system 845 estimatesthe angular orientation of the user's eye. The orientation of the eyecorresponds to the direction of the user's gaze within the NED 805. Theorientation of the user's eye is defined herein as the direction of thefoveal axis, which is the axis between the fovea (an area on the retinaof the eye with the highest concentration of photoreceptors) and thecenter of the eye's pupil. In general, when a user's eyes are fixed on apoint, the foveal axes of the user's eyes intersect that point. Thepupillary axis is another axis of the eye that is defined as the axispassing through the center of the pupil and that is perpendicular to thecorneal surface. The pupillary axis does not, in general, directly alignwith the foveal axis. Both axes intersect at the center of the pupil,but the orientation of the foveal axis is offset from the pupillary axisby approximately −1° to 8° laterally and ±4° vertically. Because thefoveal axis is defined according to the fovea, which is located in theback of the eye, the foveal axis can be difficult or impossible todetect directly in some eye tracking embodiments. Accordingly, in someembodiments, the orientation of the pupillary axis is detected and thefoveal axis is estimated based on the detected pupillary axis.

In general, movement of an eye corresponds not only to an angularrotation of the eye, but also to a translation of the eye, a change inthe torsion of the eye, and/or a change in shape of the eye. The eyetracking system 845 may also detect translation of the eye, i.e., achange in the position of the eye relative to the eye socket. In someembodiments, the translation of the eye is not detected directly, but isapproximated based on a mapping from a detected angular orientation.Translation of the eye corresponding to a change in the eye's positionrelative to the detection components of the eye tracking unit may alsobe detected. Translation of this type may occur, for example, due to ashift in the position of the NED 805 on a user's head. The eye trackingsystem 845 may also detect the torsion of the eye, i.e., rotation of theeye about the pupillary axis. The eye tracking system 845 may use thedetected torsion of the eye to estimate the orientation of the fovealaxis from the pupillary axis. The eye tracking system 845 may also tracka change in the shape of the eye, which may be approximated as a skew orscaling linear transform or a twisting distortion (e.g., due totorsional deformation). The eye tracking system 845 may estimate thefoveal axis based on some combination of the angular orientation of thepupillary axis, the translation of the eye, the torsion of the eye, andthe current shape of the eye.

As the orientation may be determined for both eyes of the user, the eyetracking system 845 is able to determine where the user is looking. TheNED 805 can use the orientation of the eye to, e.g., determine aninter-pupillary distance (IPD) of the user, determine gaze direction,introduce depth cues (e.g., blur image outside of the user's main lineof sight), collect heuristics on the user interaction in the VR media(e.g., time spent on any particular subject, object, or frame as afunction of exposed stimuli), some other function that is based in parton the orientation of at least one of the user's eyes, or somecombination thereof. Determining a direction of a user's gaze mayinclude determining a point of convergence based on the determinedorientations of the user's left and right eyes. A point of convergencemay be the point that the two foveal axes of the user's eyes intersect(or the nearest point between the two axes). The direction of the user'sgaze may be the direction of a line through the point of convergence andthrough the point halfway between the pupils of the user's eyes.

In some embodiments, the varifocal module 850 is integrated into the NED805. The varifocal module 850 may be communicatively coupled to the eyetracking system 845 in order to enable the varifocal module 850 toreceive eye tracking information from the eye tracking system 845. Thevarifocal module 850 may further modify the focus of image light emittedfrom the display 825 based on the eye tracking information received fromthe eye tracking system 845. Accordingly, the varifocal module 850 canreduce vergence-accommodation conflict that may be produced as theuser's eyes resolve the image light. In various embodiments, thevarifocal module 850 can be interfaced (e.g., either mechanically orelectrically) with at least one optical element of the optical assembly830.

In operation, the varifocal module 850 may adjust the position and/ororientation of one or more optical elements in the optical assembly 830in order to adjust the focus of image light propagating through theoptical assembly 830. In various embodiments, the varifocal module 850may use eye tracking information obtained from the eye tracking system845 to determine how to adjust one or more optical elements in theoptical assembly 830. In some embodiments, the varifocal module 850 mayperform foveated rendering of the image light based on the eye trackinginformation obtained from the eye tracking system 845 in order to adjustthe resolution of the image light emitted by the display 825. In thiscase, the varifocal module 850 configures the display 825 to display ahigh pixel density in a foveal region of the user's eye-gaze and a lowpixel density in other regions of the user's eye-gaze.

The I/O interface 875 facilitates the transfer of action requests from auser to the console 870. In addition, the I/O interface 875 facilitatesthe transfer of device feedback from the console 870 to the user. Anaction request is a request to perform a particular action. For example,an action request may be an instruction to start or end capture of imageor video data or an instruction to perform a particular action within anapplication, such as pausing video playback, increasing or decreasingthe volume of audio playback, and so forth. In various embodiments, theI/O interface 875 may include one or more input devices. Example inputdevices include: a keyboard, a mouse, a game controller, a joystick,and/or any other suitable device for receiving action requests andcommunicating the action requests to the console 870. In someembodiments, the I/O interface 875 includes an IMU 840 that capturescalibration data indicating an estimated current position of the I/Ointerface 875 relative to an initial position of the I/O interface 875.

In operation, the I/O interface 875 receives action requests from theuser and transmits those action requests to the console 870. Responsiveto receiving the action request, the console 870 performs acorresponding action. For example, responsive to receiving an actionrequest, console 870 may configure I/O interface 875 to emit hapticfeedback onto an arm of the user. For example, console 870 may configureI/O interface 875 to deliver haptic feedback to a user when an actionrequest is received. Additionally or alternatively, the console 870 mayconfigure the I/O interface 875 to generate haptic feedback when theconsole 870 performs an action, responsive to receiving an actionrequest.

The console 870 provides content to the NED 805 for processing inaccordance with information received from one or more of: the DCA 855,the eye tracking system 845, one or more other components of the NED805, and the I/O interface 875. In the embodiment shown in FIG. 8 , theconsole 870 includes an application store 860 and an engine 865. In someembodiments, the console 870 may have additional, fewer, or differentmodules and/or components than those described in conjunction with FIG.8 . Similarly, the functions further described below may be distributedamong components of the console 870 in a different manner than describedin conjunction with FIG. 8 .

The application store 860 stores one or more applications for executionby the console 870. An application is a group of instructions that, whenexecuted by a processor, performs a particular set of functions, such asgenerating content for presentation to the user. For example, anapplication may generate content in response to receiving inputs from auser (e.g., via movement of the NED 805 as the user moves his/her head,via the I/O interface 875, etc.). Examples of applications include:gaming applications, conferencing applications, video playbackapplications, or other suitable applications.

In some embodiments, the engine 865 generates a three-dimensionalmapping of the area surrounding the NED 805 (i.e., the “local area”)based on information received from the NED 805. In some embodiments, theengine 865 determines depth information for the three-dimensionalmapping of the local area based on depth data received from the NED 805.In various embodiments, the engine 865 uses depth data received from theNED 805 to update a model of the local area and to generate and/ormodify media content based in part on the updated model of the localarea.

The engine 865 also executes applications within the NED system 800 andreceives position information, acceleration information, velocityinformation, predicted future positions, or some combination thereof, ofthe NED 805. Based on the received information, the engine 865determines various forms of media content to transmit to the NED 805 forpresentation to the user. For example, if the received informationindicates that the user has looked to the left, the engine 865 generatesmedia content for the NED 805 that mirrors the user's movement in avirtual environment or in an environment augmenting the local area withadditional media content. Accordingly, the engine 865 may generateand/or modify media content (e.g., visual and/or audio content) forpresentation to the user. The engine 865 may further transmit the mediacontent to the NED 805. Additionally, in response to receiving an actionrequest from the I/O interface 875, the engine 865 may perform an actionwithin an application executing on the console 870. The engine 865 mayfurther provide feedback when the action is performed. For example, theengine 865 may configure the NED 805 to generate visual and/or audiofeedback and/or the I/O interface 875 to generate haptic feedback to theuser.

In some embodiments, based on the eye tracking information (e.g.,orientation of the user's eye) received from the eye tracking system845, the engine 865 determines a resolution of the media contentprovided to the NED 805 for presentation to the user on the display 825.The engine 865 may adjust a resolution of the visual content provided tothe NED 805 by configuring the display 825 to perform foveated renderingof the visual content, based at least in part on a direction of theuser's gaze received from the eye tracking system 845. The engine 865provides the content to the NED 805 having a high resolution on thedisplay 825 in a foveal region of the user's gaze and a low resolutionin other regions, thereby reducing the power consumption of the NED 805.In addition, using foveated rendering reduces a number of computingcycles used in rendering visual content without compromising the qualityof the user's visual experience. In some embodiments, the engine 865 canfurther use the eye tracking information to adjust a focus of the imagelight emitted from the display 825 in order to reducevergence-accommodation conflicts.

FIG. 9A is a diagram of an NED 900, according to various embodiments. Invarious embodiments, NED 900 presents media to a user. The media mayinclude visual, auditory, and haptic content. In some embodiments, NED900 provides artificial reality (e.g., virtual reality) content byproviding a real-world environment and/or computer-generated content. Insome embodiments, the computer-generated content may include visual,auditory, and haptic information. The NED 900 is an embodiment of theNED 805 and includes a front rigid body 905 and a band 910. The frontrigid body 905 includes an electronic display element of the electronicdisplay 825 (not shown in FIG. 9A), the optics assembly 830 (not shownin FIG. 9A), the IMU 840, the one or more position sensors 935, the eyetracking system 945, and the locators 922. In the embodiment shown byFIG. 9A, the position sensors 935 are located within the IMU 840, andneither the IMU 840 nor the position sensors 935 are visible to theuser.

The locators 922 are located in fixed positions on the front rigid body905 relative to one another and relative to a reference point 915. Inthe example of FIG. 9A, the reference point 915 is located at the centerof the IMU 840. Each of the locators 922 emits light that is detectableby the imaging device in the DCA 855. The locators 922, or portions ofthe locators 922, are located on a front side 920A, a top side 920B, abottom side 920C, a right side 920D, and a left side 920E of the frontrigid body 905 in the example of FIG. 9A.

The NED 900 includes the eye tracking system 945. As discussed above,the eye tracking system 945 may include a structured light generatorthat projects an interferometric structured light pattern onto theuser's eye and a camera to detect the illuminated portion of the eye.The structured light generator and the camera may be located off theaxis of the user's gaze. In various embodiments, the eye tracking system945 may include, additionally or alternatively, one or moretime-of-flight sensors and/or one or more stereo depth sensors. In FIG.9A, the eye tracking system 945 is located below the axis of the user'sgaze, although the eye tracking system 945 can alternately be placedelsewhere. Also, in some embodiments, there is at least one eye trackingunit for the left eye of the user and at least one tracking unit for theright eye of the user.

In various embodiments, the eye tracking system 945 includes one or morecameras on the inside of the NED 900. The camera(s) of the eye trackingsystem 945 may be directed inwards, toward one or both eyes of the userwhile the user is wearing the NED 900, so that the camera(s) may imagethe eye(s) and eye region(s) of the user wearing the NED 900. Thecamera(s) may be located off the axis of the user's gaze. In someembodiments, the eye tracking system 945 includes separate cameras forthe left eye and the right eye (e.g., one or more cameras directedtoward the left eye of the user and, separately, one or more camerasdirected toward the right eye of the user).

FIG. 9B is a diagram of an NED 950, according to various embodiments. Invarious embodiments, NED 950 presents media to a user. The media mayinclude visual, auditory, and haptic content. In some embodiments, NED950 provides artificial reality (e.g., augmented reality) content byproviding a real-world environment and/or computer-generated content. Insome embodiments, the computer-generated content may include visual,auditory, and haptic information. The NED 950 is an embodiment of theNED 805.

NED 950 includes frame 952 and display 954. In various embodiments, theNED 950 may include one or more additional elements. Display 954 may bepositioned at different locations on the NED 950 than the locationsillustrated in FIG. 9B. Display 954 is configured to provide content tothe user, including audiovisual content. In some embodiments, one ormore displays 954 may be located within frame 952.

NED 950 further includes eye tracking system 945 and one or morecorresponding modules 956. The modules 956 may include emitters (e.g.,light emitters) and/or sensors (e.g., image sensors, cameras). Invarious embodiments, the modules 956 are arranged at various positionsalong the inner surface of the frame 952, so that the modules 956 arefacing the eyes of a user wearing the NED 950. For example, the modules956 could include emitters that emit structured light patterns onto theeyes and image sensors to capture images of the structured light patternon the eyes. As another example, the modules 956 could include multipletime-of-flight sensors for directing light at the eyes and measuring thetime of travel of the light at each pixel of the sensors. As a furtherexample, the modules 956 could include multiple stereo depth sensors forcapturing images of the eyes from different vantage points. In variousembodiments, the modules 956 also include image sensors for capturing 2Dimages of the eyes.

In sum, a die fabrication system integrates a active light modulationinterface layer, a photonic integrated circuit layer, and an electronicintegrated circuit layer on a substrate to form a composite backplane.The composite backplane is added to an active medium layer and a toplayer to form a composite display panel, such as a liquid crystaldisplay (LCD) panel. The composite backplane 121 includes at least threelayers that are fabricated in sequence using a standard lithographicmanufacturing process. The layers include an active light modulationinterface layer 122 that includes a set of pixelated electrodes forcontrolling groups of active media that modify the amplitude or phase ofwavelengths of light propagating through the active media. The compositebackplane also includes a photonic integrated circuit layer thatincludes light-guiding waveguides and coupling components to control themodulation and direction of wavelengths of light generated by a lightsource. The composite backplane also includes an electronic integratedcircuit layer that includes electronic bus lines and integratedelectronic circuitry for components in the active light modulationinterface layer and the photonic integrated circuit layer.

At least one technical advantage of the disclosed embodiments relativeto the prior art is that the composite backplane provides an overallcompact size, small weight, and reduced system complexity. Inparticular, the composite backplane does not require post-fabricationalignment between a photonic integrated circuit layer and an activelight modulation interface, or alignment between bottom and topsubstrates of a liquid crystal cell. Further, a distance between thephotonic integrated circuit layer and an active medium layer is greatlyreduced, which improves performance of the display panel by increasingthe light efficiency, reducing optical crosstalk, and providing greatercontrol on the emission cones generated by the light source. Inaddition, the composition of the composite backplane improves the fillfactor of a display panel due to the arrangement of the photonicintegrated circuit layer over the electronic integrated circuit layer.

1. In various embodiments, an apparatus comprises a composite backplanethat modulates light from a light source, comprising an electronicslayer disposed on a substrate, a photonics integrated circuit (IC) layerdisposed on the electronics layer that causes light from the lightsource to propagate in a first direction, and an active light modulation(ALM) interface layer disposed on the photonics IC layer controls anactive medium layer in order to control the light propagating in thefirst direction.

2. The apparatus of clause 1, where the ALM interface layer comprises aset of electrodes that modulate one or more pixels, an alignment layerbetween the set of electrodes, and at least one of an anti-reflection(AR) or a partial-reflection (PR) coating, where the set of electrodesmodulate the light propagating in the first direction by controllingcorresponding pixels.

3. The apparatus of clause 1 or 2, where the ALM interface layer furthercomprises a black matrix layer interspersed between the set ofelectrodes, the active medium layer comprises a layer of liquid crystalsas an active light modulation medium, and the alignment layer isdisposed on the black matrix layer and the set of electrodes.

4. The apparatus of any of clauses 1-2, where the photonics IC layercomprises one or more light-guiding waveguides that receive the lightproduced by the light source and perform a set of optical operations,and a set of output couplers that direct the light from the one or morelight-guiding waveguides to propagate in the first direction.

5. The apparatus of any of clauses 1-4, where the light source comprisesat least one of a light-emitting diode or a laser.

6. The apparatus of any of clauses 1-5, where the photonics IC layerfurther comprises a light-coupling component that connects the lightsource with the one or more light-guiding waveguides.

7. The apparatus of any of clauses 1-6, where the photonics IC layerfurther comprises a set of optical couplers and a set of intensitymodulators, wherein the set of optical couplers and the set of intensitymodulators direct at least a portion of the light included in a firstlight-guiding waveguide to a second light-guiding waveguide.

8. The apparatus of any of clauses 1-7, where the electronics layercomprises a first electronic circuit that controls a device included inthe ALM interface layer, a first metallic via path through theelectronics layer and the photonics IC layer that couples firstelectronic circuit to the device included in the ALM interface layer, asecond electronic circuit that controls a device included in thephotonics IC layer, and a second metallic via path through theelectronics layer that couples the second electronic circuit to thedevice included in the photonics IC layer.

9. The apparatus of any of clauses 1-8, where the electronics layerfurther comprises at least a set of electronic circuits that areconnected to the first electronic circuit or the second electroniccircuit via additional via paths, and the set of electronic circuitsprocess input data and generates a set of one or more control signalsfor the ALM interface layer or the photonics IC layer.

10. The apparatus of any of clauses 1-9, further comprising an activemedium layer disposed on the ALM interface layer comprising sets ofactive media included in a set of pixels, and a top cover layer disposedon the active medium layer, where the sets of active media modify atleast one property of the light propagating in the first direction; andeach pixel in a set of pixels independently modulates a portion of thelight propagating in the first direction.

11. In various embodiments, a display system comprises a display panelcomprising a composite backplane, including an electronics layerdisposed on a substrate, a photonics integrated circuit (IC) layerdisposed on the electronics layer that directs light from a light sourceto propagate in a first direction, and an active light modulation (ALM)interface layer disposed the photonics IC layer, an active medium layerdisposed on the ALM interface layer comprising sets of pixels includingsets of an active media, and a top cover layer, and a controller causingthe display panel to modify the light controlled via the active mediumlayer or the photonic IC layer.

12. The system of clause 11, where the top cover layer comprises atleast one of a photoalignment layer, an electrode layer, or a mechanicalsupporting layer.

13. The system of clause 11 or 12, where the light source comprises atleast one of a light-emitting diode (LED), a laser, a superluminescentLED, or a nonlinear optical source, and the photonics IC layer furthercomprises a light-coupling component that connects the light source withthe one or more light-guiding waveguides.

14. The system of any of clauses 11-13, where each pixel in the set ofpixels independently modulates at least one property of the lightpropagating in the first direction.

15. The system of any of clauses 11-14, where the ALM interface layerincludes a set of electrodes that modulate the light propagating in thefirst direction by controlling the sets of the active media, and thecontroller causes the display panel to modify the light by sendingcontrol signals to the set of electrodes.

16. The system of any of clauses 11-15, where the photonics IC layercomprises one or more light-guiding waveguides that receive the lightproduced by the light source and perform a set of optical operations,and a set of output couplers that direct the light from the one or morelight-guiding waveguides to propagate in the first direction, where thecontroller causes the display panel to modify the light by sendingcontrol signals to at least one of the one or more light-guidingwaveguides or the set of output couplers.

17. The system of any of clauses 11-16, where the display panel performsamplitude modulation on the light provided by the light source.

18. The system of any of clauses 11-17, where the display panelcomprises a holographic display that performs phase modulation on thelight provided by the light source.

19. The system of any of clauses 11-18, wherein the display panelincludes the light source.

20. The system of any of clauses 11-19, where the electronics layercomprises a first electronic circuit that controls a device included inthe ALM interface layer, a first metallic via path through theelectronics layer and the photonics IC layer that couples firstelectronic circuit to the device included in the ALM interface layer, asecond electronic circuit that controls a device included in thephotonics IC layer, and a second metallic via path through theelectronics layer that couples the second electronic circuit to thedevice included in the photonics IC layer.

Any and all combinations of any of the claim elements recited in any ofthe claims and/or any elements described in this application, in anyfashion, fall within the contemplated scope of the present invention andprotection.

The descriptions of the various embodiments have been presented forpurposes of illustration, but are not intended to be exhaustive orlimited to the embodiments disclosed. Many modifications and variationswill be apparent to those of ordinary skill in the art without departingfrom the scope and spirit of the described embodiments.

Aspects of the present embodiments may be embodied as a system, methodor computer program product. Accordingly, aspects of the presentdisclosure may take the form of an entirely hardware embodiment, anentirely software embodiment (including firmware, resident software,micro-code, etc.) or an embodiment combining software and hardwareaspects that may all generally be referred to herein as a “module,” a“system,” or a “computer.” In addition, any hardware and/or softwaretechnique, process, function, component, engine, module, or systemdescribed in the present disclosure may be implemented as a circuit orset of circuits. Furthermore, aspects of the present disclosure may takethe form of a computer program product embodied in one or more computerreadable medium(s) having computer readable program code embodiedthereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

Aspects of the present disclosure are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine. The instructions, when executed via the processor ofthe computer or other programmable data processing apparatus, enable theimplementation of the functions/acts specified in the flowchart and/orblock diagram block or blocks. Such processors may be, withoutlimitation, general purpose processors, special-purpose processors,application-specific processors, or field-programmable gate arrays.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

While the preceding is directed to embodiments of the presentdisclosure, other and further embodiments of the disclosure may bedevised without departing from the basic scope thereof, and the scopethereof is determined by the claims that follow.

What is claimed is:
 1. An apparatus comprising: a composite backplanethat modulates light from a light source, comprising: an electronicslayer disposed on a substrate; a photonics integrated circuit (IC) layerdisposed on the electronics layer that causes light from the lightsource to propagate in a first direction; and an active light modulation(ALM) interface layer disposed on the photonics IC layer controls anactive medium layer in order to control the light propagating in thefirst direction.
 2. The apparatus of claim 1, wherein the ALM interfacelayer comprises: a set of electrodes that modulate one or more pixels;an alignment layer between the set of electrodes; and at least one of ananti-reflection (AR) or a partial-reflection (PR) coating; wherein theset of electrodes modulate the light propagating in the first directionby controlling corresponding pixels.
 3. The apparatus of claim 2,wherein: the ALM interface layer further comprises a black matrix layerinterspersed between the set of electrodes, the active medium layercomprises a layer of liquid crystals as an active light modulationmedium; and the alignment layer is disposed on the black matrix layerand the set of electrodes.
 4. The apparatus of claim 2, wherein thephotonics IC layer comprises: one or more light-guiding waveguides thatreceive the light produced by the light source and perform a set ofoptical operations; and a set of output couplers that direct the lightfrom the one or more light-guiding waveguides to propagate in the firstdirection.
 5. The apparatus of claim 4, wherein the light sourcecomprises at least one of a light-emitting diode or a laser.
 6. Theapparatus of claim 4, wherein the photonics IC layer further comprises alight-coupling component that connects the light source with the one ormore light-guiding waveguides.
 7. The apparatus of claim 4, wherein thephotonics IC layer further comprises a set of optical couplers and a setof intensity modulators, wherein the set of optical couplers and the setof intensity modulators direct at least a portion of the light includedin a first light-guiding waveguide to a second light-guiding waveguide.8. The apparatus of claim 1, wherein the electronics layer comprises: afirst electronic circuit that controls a device included in the ALMinterface layer; a first metallic via path through the electronics layerand the photonics IC layer that couples first electronic circuit to thedevice included in the ALM interface layer; a second electronic circuitthat controls a device included in the photonics IC layer; and a secondmetallic via path through the electronics layer that couples the secondelectronic circuit to the device included in the photonics IC layer. 9.The apparatus of claim 8, wherein: the electronics layer furthercomprises at least a set of electronic circuits that are connected tothe first electronic circuit or the second electronic circuit viaadditional via paths, and the set of electronic circuits process inputdata and generates a set of one or more control signals for the ALMinterface layer or the photonics IC layer.
 10. The apparatus of claim 1,further comprising: an active medium layer disposed on the ALM interfacelayer comprising sets of active media included in a set of pixels; and atop cover layer disposed on the active medium layer, wherein the sets ofactive media modify at least one property of the light propagating inthe first direction; and each pixel in a set of pixels independentlymodulates a portion of the light propagating in the first direction. 11.A display system comprising: a display panel comprising: a compositebackplane, including: an electronics layer disposed on a substrate, aphotonics integrated circuit (IC) layer disposed on the electronicslayer that directs light from a light source to propagate in a firstdirection, and an active light modulation (ALM) interface layer disposedthe photonics IC layer, an active medium layer disposed on the ALMinterface layer comprising sets of pixels including sets of an activemedia, and a top cover layer, and a controller causing the display panelto modify the light controlled via the active medium layer or thephotonic IC layer.
 12. The system of claim 11, wherein the top coverlayer comprises at least one of a photoalignment layer, an electrodelayer, or a mechanical supporting layer.
 13. The system of claim 11,wherein: the light source comprises at least one of a light-emittingdiode (LED), a laser, a superluminescent LED, or a nonlinear opticalsource; and the photonics IC layer further comprises a light-couplingcomponent that connects the light source with the one or morelight-guiding waveguides.
 14. The system of claim 11, wherein each pixelin the set of pixels independently modulates at least one property ofthe light propagating in the first direction.
 15. The system of claim14, wherein: the ALM interface layer includes a set of electrodes thatmodulate the light propagating in the first direction by controlling thesets of the active media; and the controller causes the display panel tomodify the light by sending control signals to the set of electrodes.16. The system of claim 11, wherein the photonics IC layer comprises:one or more light-guiding waveguides that receive the light produced bythe light source and perform a set of optical operations; and a set ofoutput couplers that direct the light from the one or more light-guidingwaveguides to propagate in the first direction, wherein the controllercauses the display panel to modify the light by sending control signalsto at least one of the one or more light-guiding waveguides or the setof output couplers.
 17. The system of claim 11, wherein the displaypanel performs amplitude modulation on the light provided by the lightsource.
 18. The system of claim 11, wherein the display panel comprisesa holographic display that performs phase modulation on the lightprovided by the light source.
 19. The system of claim 11, wherein thedisplay panel includes the light source.
 20. The system of claim 11,wherein the electronics layer comprises: a first electronic circuit thatcontrols a device included in the ALM interface layer; a first metallicvia path through the electronics layer and the photonics IC layer thatcouples first electronic circuit to the device included in the ALMinterface layer; a second electronic circuit that controls a deviceincluded in the photonics IC layer; and a second metallic via paththrough the electronics layer that couples the second electronic circuitto the device included in the photonics IC layer.